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Tvo studies were conducted to assess the listening 
ability of 17-year-old students with particular interest in^ the 
^relationship anong listening ability, verbal ability^ and s 
racial/ethnic bias in the test items. The first study, a National 
Assessment of Educational Progress and Speech Cosnunication 
Association pilot project, indicated that there vas potential it^m 
lias based on minority ethnic group response, Bovev^r, no real ddta 
on the students* verbal ability was available to substantiate 
possible explanations. In the second study, two packages- of listening 
items, each approxinately 35 ainutes_.in.Aength_^w€re^ 
25-itea""vocatular"y^ test was*^ a to estimate verbal ability. 

Recording of the listening items was lade by individuals using 
network English, The results indicated that there vas a positive 
correlation t^ween listfening ability" and verbal ability, suggesting - 
^that the dij^ering responses of minority and nonminority students in 
the first "study^could be attributed to differing levels of verbal 
ability, rather than item bias, "(MKM) 
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Introduction - ' ^ 

— -~4>»-^f-tt«-1iVortant asptcts-of coMOnlcatl 1$ listening 

«blilty. This is a caipltx s«t of skills which requlrs both cartful rt- 
call «id critical analysis of spokan Mssasts. In a socltty uhlch Is 
barrafl<rtiirtth"pemi^TlH tch<tt 

_^ .. - . - ■ ^ 1 — ■■ ■ — r'^Tf" ~ ~" 

' ' -f 

Stan, critical that ifsttning ability b« an 1aportant~toctis of Ctaminlca* 
tlon rtstareh* Instructlonf and asstssnmt. Tht prasant paptr dlscussas 
.attaopts to asstss tha listening- ability of stventeaft-ytar-olds.' It (its- 
cribes two studies which developed and field tested measures In this area 
and sane of the prbblans and conclusions which were reached concerning 
the assessMAt of listening ability. Of particular interest was the rela- 
tionships anong listening abllltyi^ verbal- ability and racial/ethnic bias In 
test itans. 

Soeakino and Listening Assessment Pilot Project 
" In June of 1976 the National Afsassmant of Educatlmal Progress (NAEP) 
and the Speech panaunl cation Association (SCA) Initiated a pilot study to 
test the feasibility of assessing speaking and listening skills (Head. 
1977a* 1977b). The products of this effert were Intended for use In'the 
National Assessment of Educational Progress* a national survey of student 
achievement with respect to Important educational objectives* funded by the 
National Center for Education Statistics. 

There are some Important difference between rutlonal Assessment and 
standardl2edjchi«y«!ient JlieJtams developed by HAEP 

measure specific objectives which are, considered Important by educators 
and content specialists'. They do not constitute a test per se. The Items 



art usad to duciibt tht tccoopllshntnts of nationally raprasantatlva 

Sroups of studahts. ^ntay ara not usad to diffarentlata lavals of ability 
monq individual stiidants. Navarthalass* tha task of davaloping 11'st«i-~ 
1n9 assassacnt ItoMS was sliitlar to standantlgri j^t^jayalopwant In tAat. . 
ft Inyolyjad^daflnlng tha dOMl n of 1 1 stani ng abl 1 1 ty and constructing 
ItaMS Nhich nasurid tint domln. ■ 

As a part of this stud/t a pooT of flfty^slx llstanl.ng Itaas was 
davalopad. Tha 1t«s wara padcagad into four tast boofclats. aach rapra- 
saf^tlng approxiMtaly ftftaan nlmitas of tasting. Tha Itaaa wara flald 
tasted In four sites which rtprasentad a varied of regions of the country t 
size and type of cities*. and racial and ethnic populations. An average of 
140 students responded to each set of itans. ^ , 

The ftsMS were analyzed using typical 1t« ana1ys1s>ftat1st1cs. Itaai 
. dlfffcuT^ was Indicated by the percent of students choosing each option. 
Itan discrlnrinatlon was Indicated by the* point btserlal correlation betwaan 
individuals choosing an option and thelr.total test scores, addition, 
tha responses to aach option were correlated with an external criterion 
Nhich /reflected the classification of tha studants as nrinorlty or non- 

V 

Mffnorlty. ^Thls added Information allowed reviewers to Identify Itaatt which 
rece1vad*s1gn1f1cant1y dl^erent responses by nlnorl^ and nonorinorl^ stu- 
dents. ■ 

It Is Inportant to anphas^ze ^e purpose of adding the extemal cri- 
terion which reflected the^ racial/ethnic background of students to the . 
Information base^ The aim of thils^strategy-was not'^-eHffl1iiate-aT1"item$~ 
which differentiated between mlnorlt/ and r.oren1nor1^ studtnts. It Is 
possible that there are real differences between these two groups with 



it 



rtsptct to 11st«n1ng ability* Ttit*1nfdnnat1on ms used to Identify Items 

- which might be discriminating betMen mlnorl^ and nonmlnorl^ students fbr 

fMsom othtr th^ llsttning skill. Fbr txanpls* an '1t«m night receive 
different responses beceuse of the varying backgroundSt experlencest values 
~ (rnins^*W Vie contldered 

;tbese factors to be extreneous to listening ability .■ 
The results of fteld testing indicated no problens wItt respect to 



selecting Item with appropr_late..d1fjf1cu1ty level and discrimination power. 
Guidelines had been ^tabllshed for selecting Items within <the difficulty 

L " ' 

range of forty percent to elgh^ percmt correct responses with an average 
of sixty percent (Stanley and Hopklhst 1972) and with a discrimination 
level, of at -least ..30 (Harrlst 1968). Because the purpose of National 
Asiessnent U not to build a test but to select items which measure spe- 
ciflc objoctlyest these guidelines-were mrely-^uggestlvc^dnrat-xrucfarh- 
Pratitlcally all of the itens In the pool met the discrimination require- 
ment and only about twent;/ percent of the items fell outside the proposed 
difficult range. 

The surprising result from.tryouts wasthat the listening Items, un- 
llice the Items for the other areas of comminlcatlon competence which were 
field testeit. at the same time, showed a high number of significant point 
biserlal correlations between responses and mlnority/nonmlnorlty status. 
Approximately one-half of the listening Items demonstrated this character- 
istic. It must be emphasized that a. significant correlation between tht 

responses-and-mlnorl^/nonmlndrlty status (a relationship significantly' 
diffisrent from zero) was not considered tantamount to item bias. There 
were a couple of reasons for reviewing the data cautiously. First, the 

s. ■ 
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— ti70ut-s1tts-1nc1ttdtd-two~tll-arfnor1ty'Schools, This n»dt it pojilblt^t- 
tlM distributions wiiht Inctudt a concantratlbn of iitiioiity studMts irtthln 
« sififfit option bocauso of son unusual rtsponsts by tho sttidtnts In ttwst 
sdMioTsT SaoMidly* iffmt mitMr of tonrtlations war* rtv1«Md» ont for 
Mdi Aril of aach 1tai» Amg thast. thara Mtra bound to ba soM signlft* 
cant ralatlonsMps du» to ctanct (iant out of tManty). 

A signlfteant eorralatlon ims centldarad a signal for fUrthar ravlait. 
In soia <easat> th* crltlqut Indleatad posslbl* sourcas of Itaa blu* such i 
u a typically lAlta spateb pattarn In a llstaning stlnull tAlch prasantad , 
a parsuasiva conyarsation batwaan tNo frlMds. and'tha 1t«i tias droppad. 
In othar casas. boMivar. th* ravlaw could not datact any prain«i and tba 
1t«a was ratalnad. As Indleatad aart1*r« a significant corrtlatldh vas not 
„considarBd^to^b*-«ynonyBOus..w1th Itas-blas. Howavar. tha fk^aquancy of this 
charaetoHstfe and tht Mritad dtfftranca bttMaan tMs sat of ItMs and tha 
othar stts of ItaM (Infomring spaalc1n9..contro111ng spaaklng. ritualizing. 

« 

and sharing dialings) su^gastad a spacial problaa.* 

A panti of spaach conKinlcatlqn'axparts ravlawad tha llstaning ItaiiB 
- and salactad appraxluataly ona-half for usa f n tha assassmant. About ona* 
third of tha salactad ItaM raflactad significantly dlffirtnt rasponsas by 
minority, and nonainorl^ studants. Tha consultants Idantlflad vaiy faw 
^spadflc aspaets of tha llstaning ItaM which thay fait wara Indlcativ* of 
■ Itw bias, such u tha typa of situation prasantad, tha spaach styla usad. 
or tha valuas^lapllad. Howavar. thay spaculatad a nunbar of ganaral char- 

♦ 

actarlstlcs' of tha ItMs which might hava tappad ftctors which wara axtra- 
naous to naasuHng llstaning ablll^. Thasa problems included: 



1) ttit vocabulaty Itvtl of the tlsttnlng stimuli; 

^2)-^th»-l tngth-orthr-ftrml-sptechesir 

3; ttM intertst levtti of th* llstttnlng stmuHi 
4) tlM «cc«nt and rate of speech of the speakers on the stimulus 
5 tnpesi end 

S^the level of disruption In the classroMS. 
They Kypoth^litftd ttut tilnorl ty studmts night have 1e5s spiKlatlzed 
voeabulatv kndMledgei a lOMr tolerance for lonji* boring naterlaUs and 
less experience listening to the accents and rate of Mhlte speakers. 
FurtheVmoret minority students might tend to be concentrated In schools 
Mhere there t*ere more disruptions In the classrooms and nearby environment. 

An additional factor which might explain the results Is vatylng levels 
of verbal ability of the mlnorl^ and nomrlnorlty students In the tryout 
groups. If listening ablll^ overlaps with verbal abllltyt as previous re- 
search IHIcates. H Is possible that the results might be partially ex- 
plained In terms of different levels of vertal ability* The field testing- 
did not collect Information about the verbal ability of t^e students. It 
Is possible that the minorl^ students selected fOr ttyouts reflected an 
overall lower level of verbal ability than. the nonminarlty students. 

The outcone of the ttyout phase o^ the pilot listening assessment was 
the Identification of a problem* potential Item blast and no real data te 
substantiate or ^rther elaborate the situation. A numberV explanations 
of the n»ults were proposed. However, the^e explanations were based on 
speculation and not on empirical evidence. The problem of minority bias 
,had not^n^clMrly articulated 1n^ pas t^^l^^^ 

results af~the-NAEf>/SCA-p11ot~project-^uggested-a-clear-need for further 
development' and research. ' 



i>1«t«t1iKi^At$M$a>fit fbllow-Up-Study 

A llsttfting ustssawit fo11oM-up stu4y'wu conduettd'>taa1)«vt«tt tht 
|>rQb1«n IdMtlfltd by th* m1«wrs of tii* prtvlous «tu4y and to txplort-^ 
■or* dtflnltlvtly th«.:irt1at1«nsli1^- aaong llsttnlng «b1,11ty. .v«r1»1 ability, 
and rae1a1/«tliiric bias. IVm |»aekag«* of llstanlifs ItaMS. aach approxiMttly 
3S irinutas In langth, Ntra davalopad ft*11ow1ng tha susgastlons of tha ra-' 
vlawars. SoM MtaHaU frw tha aarllar $tu4y rawHttan and othttrs 
Mra addad. Tha final sat of AattrlaU Included fivt Infonaal Infomatlva 
llstaning situations*^ nawseast. a public sarvica announciaiaRt. a liwathar * 

report. a-ta1ephone call and e public address amiounc«Mnt«4wd- two Infbr- 

* ■ f 

oal persuasive listening situations— a paid political announcMant and a 
coMerclal.. Also Included i«era two speeches, one about poTtutlon and 
another about aTlan workers, both wi th persu asive elaaants. On* Interactive^ 
situetlon wu Included, an Interview between an aaplgyer and a teenage Job 
seeker. These Mterlals were kept purposefully short, all within the range ^ 
of thirty -seconds to three Minutes.' We used the Oale^all and fry read- 
ability fbnnulas to nauure tha -difficulty of tha naterlal and found that 
the readability of the materials ranged frm fifth grade to college level. 

♦ 

Questions, about the llstenlng-stlmll were developed which reflected^ 
five types of listening tasks: 

1j to be able to recall significant datallsi 

2) to be able to coaiprehend the naln Idea; 

3) to be able to draw Inferences about the Informtlon 
(e.g.. relationships* Implications); 

4) to be able to isake iudgments concamlng the speaker — 
^l*gjL*Jlntent»-a.nttudas),;rand l. 

5) to be able to nake judgments concerning the Information 
(e.g.. types of evidence, logic of ergumcnts)'. 

Recordings of the materials were made by Individuals using network English. 
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' to tiM cr1t1c1s«$ of reviewers. Each of the.tw oackagts of matertilS was 
^fitio tested with four classroons of eleventh graders In four sites, repre- 
senting four .parts of the country, and a vart«ty-of sizes and types of city. 
There were a total of 84 rdlpondents for package one'and 86 fbr package iMoi 

In order to explore the relationship among listening ability, vertal 
ability unA racial/ethnic bias, additional data were collected. All stu- 
dents responded to a 25»1taB vocabulary test, a subtest of .the Lorge- 
Thorndlke Intelligence Tests (lorge. Thomdike and. Hagen. 1966). This was 



used to estimate the verbal ib^lity of the students. The respondents ^to 
package one Included s1?t percent nrinorlty students and the respondents to ■ 
package two included fSwrteen perce'ht mtnortty-students. The nrtnorlty 
representation was lower than expected, ftevertheless, the results still 

' : ^ -s— 

, Indicated some Inter^tlng relationships. - 

The results of the field test showed that the two packages of materials 
were perfoming similarly. The mean listening -score for the first package 
wa^ 59.5 percent correct witn' a standard deviation of 4.86. The mean score' 
for the second package was 55.9 percent correct with a standard deviation 

/ 

Of 5.51. The students who responded to the two sets of , Items were similar 
In verbal ability. The students who took package one had a mean vocabulary 
iscore of 11.77 and the students who took package two had a mean vocabulary 
score of 10.66. An Interesting contrast In the two groups was that the 
wan vocabulary score of the minority students responding to package one 
was 12,8 and the mean, vocabulary score of the minority students responding 
to package two was 6,7. Thus, the minority students responding to package 
one wereabove^e average for -their group*-and-the minority students 

-7- ' - ' ' 
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rtsponding to' packagt two Mint V«tT1 btlov ttw «v«rast of thtir group. 

fa irtg ht'b* txptcttd i n a fitid tait. m 1wtjyj<tetj_ltw <HtHtl<^ 
Indlcatad that sam itm mri clurly not <9ant1n9 wtll. Uith mp«ct to 
ItM difftculty* tM iima in ptdcaQt on« and nlna In package ^ fafltd to 
fall iHthin th* dtilrad rang* of forly^ ptrcott to tight/ ptretrr^ coVrtct 
(Stanley and HopMm* 1972). Ulth rtsiitct to Itaa dlscrlnlnatton* jtlxttan 
1t«B In padtagt ont and six In pactiagt two falTt^ to MMt tbt dtslrtd 1ta»- 
total coirtlatlon of .30 (Harris-. 1968). Uhllktt tht i«suTts in tttt first' 
stud/* tht ItMS In thtst tM padcagts did not show significantly difftrtnt 
rtsponsts for mliiorlty and nomrinorlty studtnts. Only ont Itan displayed 
this charactwistlc. It sttned citar that froR tht Itan pool It twultt bo 
posslbit to plck.adtquatt stlowll and iUm to ust In ont finallztd packagt. 

Em w ith tht_pj»fcTa«-w!th Individual -1t«»-c1ttd abov<. tht Ittw-ai- 

a wtwlt apptaM to b« optratlng quit* wall. Tht rtllablllty of Itats in 
packagt ont was .78 and In packagt two was .73 (using Hp/t's ntasurt of 
Inttmal conslsttncy). It. thtnfPrt. sttntd approprlatt to procttd to Took 
at'tht Inttrrtlatlonships aaong llsttnlng abllltyt vtital ablli^ snd 
radal/tthnlc bias. It tan d- curious that tht preblan of posslbit racial/ 
tthnic bias which had fattn so prtyaltnt In tht first study was not tvldent 
In tht stcond s«t of Itvs. Ir-^ttd. steps had bttn taktn to allavlatt tht 
possible problem of bias by controlling tht vocabulary level, decreasing 
length t Incrtlsing Inttrtst and using bttttr tap* rtcordlngs. Howtvtr. tht 
Inftimtlon conctrnlng tht v trbal ablH^ of tht^ajd^ttetjj^ps added 
Inportant additional evidence for explaining tht results. . 

Tablts 1 and 2 provide tht Inttrcorrelatlons anmg listening adiljtyt 
vtital ability .and minority status for the respondents of packages one and 



TMCE 1 

tnttreorrt^Atloits Anong Usttnlng. Ability. 



LlstMlng ^ Vtrbal Nlnorlty. 

Ability . Ability Status 



Ustming Ablllt/ ><0 *S9t* 
■ViHrtMtl Ability 1.0'. .'052 
mnorltyStitus . . 1.0 



TABLE 2 . 

tnttrcorriUtloftt AMfig Llsttnln^ Ability. 
Vtrtal Ablllt/ 9ni NlnoHt/ Status for Padctgt Tm 





Listtning , 


Vtrtal 


■ Minority 




■ Ability 


Ability 


Status 


Usttnlng Ablllt/ 


1.0 , 


.474* 


-.232** 


Vtrtal Ability 




1.0/^ 


-.255** . 


Nlnorlty Status 









>P < .01 
*^ < .05 

tuo* It Is citar that tlw relationship bttMstn Usttnlng ability %nd vartal 
ability for both stts of 1t«is vms high* .59 correlations for package ont 
and .47 for padcagt tVM. this is in Una irtth'tht r^ults of previous stud* 
les of this relationship (Crook. 1d57i Haberland. 1959). Hie differences In 
the results of the tiM pedcages vns Indicated by the relationships ietvwen 
the factors of llstenlng'ablllty and verbal ability, and the factor of ' 
minority status. Fbr package one. there was no significant correlation be- 
tween listening ablH^ and mlnorl'ty status, nor between verbal ability and 



■ffwHty status, for pickagt two. the corrtUtlon tetutM llsttnlng ability 
and B lnorlty status was -.23 vnd btttmn -ve rtel ability and KfiMrl^j^tus 

- _ _ . . — . ^v,> :r ■• — — 

.26 ' Tfitst corralatlont wart of slKfla^ mgnitudt and '1nd1atnir:4l9«> 
niflcantly dlffiaraitt *0S) rtsponsas of jafnorlty ami 

dants. Tim Minority studanta Mrt ptrfbnritot btlow tha Itvcl of ttw non- 

' , ■ * • 

Vvfncrtty stbdants on both Maturts. It stMuld tm <w11td tliat tha vajtel 
ability of tbt KlnoHty studonts Mho raspondad to package ona was abova 
thair group avtraga* and tha varbal ability of tha Minority studanta vho 
rtspdndM to pacJcaga two ms «ia11 balow thair group avarago. This cohac-" 
tlvt tvldanca saMS ta Indlata that tha differing parforntuioa of Minority 
students on the listening aeesures my be attributed ta diffiiring levels 
of vertial ability rather than to r^cTalTeSinlc bias within the 1ta».th«i- 
selm. - ' / . - ■ ■ . 

• trtycus'slwi 

The results of- the -second listening stody confirm earlier sttidles're* 
garding a positive correlation between listening ablllty'and vert}a1 ability. 
It also Mggests'that differing responses of nrinorlty and nonrlnorlty stu- 
dents to listening itaas in the first study nay be attributed to differing 
levels of verbal ability In the grqup and not to problans of racial/ethnic ' 
blu In the iteto. tn retrospect* this seaas a reaso)iab1| explanation ^ 
considering the saJte up of the first field test saaple. In. the first study* 
the nbnarinorlty students Mstly cane from snail dtles with generally afflu- 
ent school populations^" The iinorlty studaits oostly cane from 1arge~ ~ 
dtles with generally poor school populations. However, the second study 
does not fully explain whV In the original study tha dIffirencK In 

-10- ^2. 



responses of minority and normlnorlty students were so much greater with 
the listening Items^ than with tha Items which were assessing other conmunl- 



cation canpetencieso-lnformatlve speiJclng. persuasive speaking, ritualizing 
and sharing feelings. There Is a suggestion fron the two studies that 
listening ability may be tied nore directly to veital ability then other 
cOMDunlcatlon competencies. This finding presents Interesting possibilities 
for further study. - " ■ . ' 
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